Multiple Testing for Gene Expression Data: An Investigation of Null Distributions with Consequences for the Permutation Test

Gene expression studies produce data with which inferences can be made for thousands of genes simultaneously, allowing researchers to answer questions such as “Which genes are significantly differently expressed between two (or more) conditions?” or “Which genes have a significant association with an outcome or covariate?”. In order to make statements about the statistical significance of thousands of genes at once, it is essential to appropriately account for multiple tests. Multiple testing methods are hypothesis testing procedures designed to simultaneously test p > 1 hypotheses while controlling an error rate. Traditional approaches to the multiplicity problem are reviewed by [1]. More recent developments in the field include resampling methods [2], stepwise procedures, and the false discovery rate [3]. Standard practice in multiple testing with gene expression data is to use t-statistics as test statistics and to control a type I error rate under the permutation distribution. In this paper, we revisit the rationale behind such choices and suggest situations in which alternatives are more sensible. 2 Multiple Hypothesis Testing Procedures